- **Inference** – Platforms and engines for running models, plus user interfaces.
- **Models** – LLMs (general, coding, multimodal, image, audio), model providers, and specific model highlights.
- **RAG** – Retrieval-Augmented Generation tools.
- **Safeguards** – Safety and content filtering.
- **Agents & Tools** – Agent frameworks, Model Context Protocol, coding agents, computer/browser automation, memory management, and testing/evaluation.
- **Research, Training & Fine-tuning** – Security, sandboxing, and model development.
- **Hardware** – Local hardware options.
- **Tutorials** – Guides covering models, prompt/context engineering, inference, agents, and RAG.
- **Communities** – Places to connect and share knowledge.
- **Models** – LLMs (general, coding, multimodal, image, audio), model providers, and specific model highlights.
- **RAG** – Retrieval-Augmented Generation tools.
- **Safeguards** – Safety and content filtering.
- **Agents & Tools** – Agent frameworks, Model Context Protocol, coding agents, computer/browser automation, memory management, and testing/evaluation.
- **Research, Training & Fine-tuning** – Security, sandboxing, and model development.
- **Hardware** – Local hardware options.
- **Tutorials** – Guides covering models, prompt/context engineering, inference, agents, and RAG.
- **Communities** – Places to connect and share knowledge.
El Assadi et al. compare ten LLMs (six families) and 26 embedding models (118M - 14B parameters) on 37 tasks, considering cost. In aggregate, the two paradigms are effectively tied (best LLM scores 77.6 versus best embedding model 77.2), yet their strengths diverge by task: LLMs lead on reasoning-heavy retrieval while embedding models lead on classification, and the two match on clustering, STS, and pair classification.
LLMs are significantly more expensive (up to 1,431x) and slower (2.5-736x) than embedding models for certain tasks. The authors suggest using embedding models for similarity, classification, and clustering, and LLMs for reasoning in retrieval.
Reasoning tokens are 28-81% of LLM inference cost; lower budgets maintain or boost retrieval quality for most tested models.
- Only Gemini 3.1 Pro breaks into the Pareto frontier alongside the leading embedding models.
- Accepted to COLM 2026; code, datasets, and results are publicly released on GitHub.
LLMs are significantly more expensive (up to 1,431x) and slower (2.5-736x) than embedding models for certain tasks. The authors suggest using embedding models for similarity, classification, and clustering, and LLMs for reasoning in retrieval.
Reasoning tokens are 28-81% of LLM inference cost; lower budgets maintain or boost retrieval quality for most tested models.
- Only Gemini 3.1 Pro breaks into the Pareto frontier alongside the leading embedding models.
- Accepted to COLM 2026; code, datasets, and results are publicly released on GitHub.
simongonzalezdc writes about a serving-side tool-attention failure in Qwen3.8-27B via llama-server (jinja templates) where the position of tools in the rendered payload determines whether the model will invoke them. With an 8-tool list, early-positioned, description-less tools were verbally refused as "not wired up" (0/6 success), while the same tools worked perfectly when moved to the tail (3/3) or given a one-line description (6/6). A follow-up with 13 tools confirmed the pattern: a mid-list pair prescribed by name failed 0/5, but tail placement restored 3/3 clean invocation.
- Hypothesis: later tools in the rendered template output receive more attention; compact single-JSON-blob rendering may amplify the position effect versus per-tool natural-language rendering.
- Controlled matrix (24 runs + 27 curl controls) ruled out tool count, context/style, effort level, and fresh-session as causes.
- Test hardware: AMD Strix Halo (gfx1151/HIP) running Q4_K_XL quantization on current llama-server.
- Full methodology and ledgers published in a companion repo: KyaniteLabs/qwen38-27b-strix-halo.
- Hypothesis: later tools in the rendered template output receive more attention; compact single-JSON-blob rendering may amplify the position effect versus per-tool natural-language rendering.
- Controlled matrix (24 runs + 27 curl controls) ruled out tool count, context/style, effort level, and fresh-session as causes.
- Test hardware: AMD Strix Halo (gfx1151/HIP) running Q4_K_XL quantization on current llama-server.
- Full methodology and ledgers published in a companion repo: KyaniteLabs/qwen38-27b-strix-halo.
paoloap wrote companion code for a Medium article on agent harnesses, including five layers: execution boundary, sandboxing, memory persistence, verification loops, and context pipelines, each with failure and guard scripts running alongside.
Move enforcement out of system prompts to deterministic code; a rule like "never delete without human approval" becomes a pre-execution hook denying the call, not a sentence the model might forget.Every demo runs with no API key; the model is replaced by a ~40-line scripted stand-in that emits a fixed sequence of tool calls, making each failure reproducible and the CI suite able to assert behaviour rather than smoke-test it. A single dependency-free `harness.py` can be dropped into any existing agent loop to add boundary checks, path allowlists, hostname allowlists, a persistent store, read-only review, dry-run, and token-cost distillation.
- `boundary()` detects coroutine functions and awaits them, fixing a silent no-op that made the guard a no-op on the most common (async) agent loops
- `host_allowlist` compares the parsed hostname, so `api.openai.com.evil.com` and `evil.com/?x=api.openai.com` are both refused
- `Denied` subclasses `str` so it drops into the same slot a tool result occupies, keeping existing loops unchanged
- `boundary(rules, max_repeats=3)` detects identical consecutive denials and changes the message to break a deterministic deadlock at full token cost
- The repo is MIT-licensed, Python 3.8+, and has no external dependencies
Move enforcement out of system prompts to deterministic code; a rule like "never delete without human approval" becomes a pre-execution hook denying the call, not a sentence the model might forget.Every demo runs with no API key; the model is replaced by a ~40-line scripted stand-in that emits a fixed sequence of tool calls, making each failure reproducible and the CI suite able to assert behaviour rather than smoke-test it. A single dependency-free `harness.py` can be dropped into any existing agent loop to add boundary checks, path allowlists, hostname allowlists, a persistent store, read-only review, dry-run, and token-cost distillation.
- `boundary()` detects coroutine functions and awaits them, fixing a silent no-op that made the guard a no-op on the most common (async) agent loops
- `host_allowlist` compares the parsed hostname, so `api.openai.com.evil.com` and `evil.com/?x=api.openai.com` are both refused
- `Denied` subclasses `str` so it drops into the same slot a tool result occupies, keeping existing loops unchanged
- `boundary(rules, max_repeats=3)` detects identical consecutive denials and changes the message to break a deterministic deadlock at full token cost
- The repo is MIT-licensed, Python 3.8+, and has no external dependencies
Ory Team states traditional IAM frameworks (MFA, SSO, fixed API keys, IP whitelisting) are insufficient for autonomous AI agents entering production, as these agents reason, use tools, and execute multi-step workflows without constant human intervention.
The article describes six identity capabilities for securing these agents: verifiable cryptographically-signed agent identities with delegation chains, just-in-time ephemeral credentials, relationship-based access control bound to task intent, machine-speed automated containment via circuit breakers, in-the-loop runtime policy enforcement with configurable human approvals, and a web-scale identity control plane that handles machine-speed throughput and rapid sub-agent lifecycle governance.
- Sponsored post by Ory; Insight Partners (TNS owner) is an investor in both Ory and TNS.
- Ken Buckler (EMA Research Director) is quoted: "most organizations are woefully unprepared" for the security risks of managing agentic identities.
- A comparison table contrasts agents with humans and service accounts across velocity, decision logic, auth mechanics, and access granularity, highlighting that agents need ephemeral delegation and contextual attestation rather than passkeys or static keys.
- The ReBAC example given: "Agent X may read Document Y only if human user Z is the document owner and the active workflow is 'Data Summarization'."
- PKCE and strict token-binding are called out to prevent credential replay outside the agent's intended runtime context.
The article describes six identity capabilities for securing these agents: verifiable cryptographically-signed agent identities with delegation chains, just-in-time ephemeral credentials, relationship-based access control bound to task intent, machine-speed automated containment via circuit breakers, in-the-loop runtime policy enforcement with configurable human approvals, and a web-scale identity control plane that handles machine-speed throughput and rapid sub-agent lifecycle governance.
- Sponsored post by Ory; Insight Partners (TNS owner) is an investor in both Ory and TNS.
- Ken Buckler (EMA Research Director) is quoted: "most organizations are woefully unprepared" for the security risks of managing agentic identities.
- A comparison table contrasts agents with humans and service accounts across velocity, decision logic, auth mechanics, and access granularity, highlighting that agents need ephemeral delegation and contextual attestation rather than passkeys or static keys.
- The ReBAC example given: "Agent X may read Document Y only if human user Z is the document owner and the active workflow is 'Data Summarization'."
- PKCE and strict token-binding are called out to prevent credential replay outside the agent's intended runtime context.
Michael Nuñez writes about Slack Code, a new product that embeds AI coding agents'', including Anthropic's Claude Code, Cognition's Devin, GitHub Copilot, and Vercel's agent'', into dedicated Slack channels where entire teams can watch, steer, review, and ship software together. The pitch reframes AI coding from a solitary terminal exercise into a "multiplayer" newsroom workflow: tagging an agent spawns a project-specific channel with live code diffs, previews, and a running plan, then archives into a searchable audit trail. Slack argues that with generation no longer the scarce step, the bottleneck has shifted to human judgment, taste, and craft work any team member can contribute to in a shared channel.
- Cognition reported internal merged PRs up 10x while headcount rose only 40%, with engineers simultaneously managing "dozens of agents."
- Security model: agents inherit the invoking user's ACLs with no bot-level or god permissions; Devin sandboxes run with "minimum viable access" and an optional
- Cognition reported internal merged PRs up 10x while headcount rose only 40%, with engineers simultaneously managing "dozens of agents."
- Security model: agents inherit the invoking user's ACLs with no bot-level or god permissions; Devin sandboxes run with "minimum viable access" and an optional
Anurag Singh replaced five Python scripts (backup, organizer, renamer, cleaner, watchdog) with a local LLM agent, which made errors the scripts didn't (wrong directories, skipped steps, false success reports).Each of the original scripts followed explicit rules through a scheduler; the agent instead added a longer inference chain (inspect, interpret, choose a tool, build a command, execute, review) to tasks that fixed logic already described completely, while also holding a loaded model in memory between runs.
- AutomationBench scores for frontier models remain well under 20%: GPT-5.6 Sol 18.1%, GPT-5.5 12.9%, Claude Opus 4.8 15.5%, Gemini 3.5 Flash 14.5%
- Granting an LLM system-level access creates a prompt-injection vector: a malicious file on disk could carry instructions the agent interprets as commands
- Singh's proposed fix: let the agent classify and route ambiguous requests, then hand off to a validator + fixed script for the actual filesystem action
- The five original scripts covered photo backup, extension-based Downloads sorting, file renaming, app-cache clearing, and a disk-threshold alert
- AutomationBench scores for frontier models remain well under 20%: GPT-5.6 Sol 18.1%, GPT-5.5 12.9%, Claude Opus 4.8 15.5%, Gemini 3.5 Flash 14.5%
- Granting an LLM system-level access creates a prompt-injection vector: a malicious file on disk could carry instructions the agent interprets as commands
- Singh's proposed fix: let the agent classify and route ambiguous requests, then hand off to a validator + fixed script for the actual filesystem action
- The five original scripts covered photo backup, extension-based Downloads sorting, file renaming, app-cache clearing, and a disk-threshold alert
This XDA Developers article by Anurag Singh explains how a **CLAUDE.md** file at the root of a repository solves the problem of Claude Code repeatedly asking the same setup questions in every new session.
**The problem:** Each Claude Code session starts with a fresh context window, so it has no memory of previous conversations. It must re-inspect the repo and re-infer project conventions (package manager, test commands, directory rules, etc.), wasting time and tokens—and sometimes reaching different conclusions.
**The solution:** A `CLAUDE.md` file that Claude Code automatically loads at the start of every session. It acts as a persistent onboarding document containing:
- **Commands** (e.g., "Use pnpm," "Run `pnpm test` before completing a task")
- **Project structure rules** (e.g., "Reusable components go in `src/components/`," "Do not edit `src/generated/`")
- **Working rules** (e.g., "Reuse existing components," "Ask before installing a dependency," "Make the smallest change required")
**How to create it:** Either write it manually or run `/init` inside Claude Code, which auto-generates a starting file from the repo. If one already exists, `/init` suggests changes rather than overwriting.
**Best practices:**
- Keep it under ~200 lines (treat as a ceiling, not a target).
- Be specific—avoid vague instructions like "write clean code."
- Don't duplicate content Claude can discover by reading the repo (don't make it another README).
- Watch for conflicting rules across multiple instruction files.
**File hierarchy:**
| File | Scope |
|---|---|
| `~/.claude/CLAUDE.md` | Global, all projects |
| `CLAUDE.md` (repo root) | Project-level, commit to version control |
| `CLAUDE.local.md` | Personal, add to `.gitignore` |
The author notes that Claude Code's built-in "auto memory" is unreliable for critical rules because Claude decides what to save there; a hand-written CLAUDE.md is exact and shareable.
**The problem:** Each Claude Code session starts with a fresh context window, so it has no memory of previous conversations. It must re-inspect the repo and re-infer project conventions (package manager, test commands, directory rules, etc.), wasting time and tokens—and sometimes reaching different conclusions.
**The solution:** A `CLAUDE.md` file that Claude Code automatically loads at the start of every session. It acts as a persistent onboarding document containing:
- **Commands** (e.g., "Use pnpm," "Run `pnpm test` before completing a task")
- **Project structure rules** (e.g., "Reusable components go in `src/components/`," "Do not edit `src/generated/`")
- **Working rules** (e.g., "Reuse existing components," "Ask before installing a dependency," "Make the smallest change required")
**How to create it:** Either write it manually or run `/init` inside Claude Code, which auto-generates a starting file from the repo. If one already exists, `/init` suggests changes rather than overwriting.
**Best practices:**
- Keep it under ~200 lines (treat as a ceiling, not a target).
- Be specific—avoid vague instructions like "write clean code."
- Don't duplicate content Claude can discover by reading the repo (don't make it another README).
- Watch for conflicting rules across multiple instruction files.
**File hierarchy:**
| File | Scope |
|---|---|
| `~/.claude/CLAUDE.md` | Global, all projects |
| `CLAUDE.md` (repo root) | Project-level, commit to version control |
| `CLAUDE.local.md` | Personal, add to `.gitignore` |
The author notes that Claude Code's built-in "auto memory" is unreliable for critical rules because Claude decides what to save there; a hand-written CLAUDE.md is exact and shareable.
Alibaba has open-sourced Qwen-UI-Agent, a GUI agent foundation model that operates across mobile, desktop, web, and deep-search environments on real hardware rather than relying on simulation. It achieves top benchmark results: 82.1% on MobileWorld, 79.5% on OSWorld-Verified, and first on WebArena. It also introduces MobileWorld-Real, a 400+ task benchmark on 100+ phones and 150+ apps, with a 92.2% success rate.
- Supports command-line execution alongside standard GUI operations and batches multiple actions into a single decision step to shorten trajectories.
- Built-in safety layer refuses illegal or high-risk requests outright and pauses at sensitive operations (payments, data deletion, privacy grants) for explicit user confirmation.
- Trained via online reinforcement learning on trajectories exceeding 100 steps, paired with adaptive curriculum learning to progressively tackle longer tasks.
- Supports command-line execution alongside standard GUI operations and batches multiple actions into a single decision step to shorten trajectories.
- Built-in safety layer refuses illegal or high-risk requests outright and pauses at sensitive operations (payments, data deletion, privacy grants) for explicit user confirmation.
- Trained via online reinforcement learning on trajectories exceeding 100 steps, paired with adaptive curriculum learning to progressively tackle longer tasks.
Thomas Joos writes that choosing LLM deployment now means picking between cloud APIs, datacenter GPUs or local workstations, with technical limits set by memory and bandwidth rather than raw compute and software choice tied to load profile. Quantization shrinks models to fit consumer GPUs and unified-memory systems, while token generation is bandwidth bound so bandwidth per dollar matters more than TFLOPS. Economically, self-hosting an eight-H100 cluster costs about $237k per year over three years versus higher on-demand cloud pricing, but break-even needs 50-83% sustained utilization that teams typically reach only 40-65%, and the strongest case for local inference is sovereignty and resilience not pure cost.
- Mistral Large 3 ships as a 675B-parameter mixture-of-experts with 41B active per token under Apache 2.0
- Nvidia DGX Spark offers 128 GB unified memory at $4,699 list price since Feb 23 2026
- vLLM generates 44x the tokens per second of llama.cpp at 64 concurrent users on H200 with Llama 3.1 8B
- DevTk finds self-hosting breaks even only at several billion tokens per month versus budget hosted open models
- Martin Kuppinger says sovereignty is not an end in itself; security and resilience come first and exit path matters more than immediate rebuild
- Mistral Large 3 ships as a 675B-parameter mixture-of-experts with 41B active per token under Apache 2.0
- Nvidia DGX Spark offers 128 GB unified memory at $4,699 list price since Feb 23 2026
- vLLM generates 44x the tokens per second of llama.cpp at 64 concurrent users on H200 with Llama 3.1 8B
- DevTk finds self-hosting breaks even only at several billion tokens per month versus budget hosted open models
- Martin Kuppinger says sovereignty is not an end in itself; security and resilience come first and exit path matters more than immediate rebuild
